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1.  General  Summary  of  Achievements. 

Grant  FA8655-09- 1-3022,  “Multisensory  Control  of  Stabilization  Reflexes“,  provided 
general  funding  for  research  in  my  lab.  It  covered  the  salary  of  one  of  my  senior 
research  associates  (RA),  Dr.  Kit  Longden,  who  works  on  energy  efficient,  state- 
dependent  coding  of  self-motion  information  (see  below).  Kit  is  now  funded  from 
another  grant  with  AFOSR/EOARD  which  I  am  holding  (FA8655-09- 1-3083) 
together  with  Dr.  Sean  Humbert  (FA9550-09-1-0075),  University  of  Maryland  on  the 
integration  of  compound  eye  and  ocellar  signals  for  gaze  and  flight  control.  In  this 
final  report  I  will  stick  with  the  tradition  I  previously  introduced  to  include  related 
developments  and  progress  on  (a)  all  Air  Force-funded  research  activities  and  (b) 
most  if  not  all  projects  supported  by  other  funding  agencies.  As  a  result,  some  of  the 
material  I  include  here  may  also  be  presented  in  other  reports  submitted  by  my 
collaborators  or  by  myself. 
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1.1  Publications 


When  I  submitted  my  last  report  two  journal  articles  (035-36)  and  a  book  chapter 
(R4)  were  only  submitted  but  have  now  been  successfully  published.  In  2011/2012 
another  four  papers  in  peer  reviewed  journals  came  out:  on  multisensory  integration  in 
the  fly  neck  motor  system  (037),  on  (fly)brain-machine  interfacing  (038),  on  state- 
dependent  processing  in  lobula  plate  tangential  cells  (039)  and  another  one  in  Journal 
of  Neuroscience  (040)  on  behavioural  and  electrophysiological  responses  of  flies  to 
“plaid”  stimuli  (superimposed  visual  gratings)  in  combination  with  phenomenological 
modelling  of  the  motion  vision  pathway. 

Total  number  of  publications  within  the  funding  period  directly  or  indirectly 
supported  by  AFRL/AFOSR/EOARD: 

•  12  publications  in  peer  reviewed  journals  (028-36,  038-40) 

•  4  peer  reviewed  conference  papers  (C3-C6) 

•  1  book  chapter  (R4) 

•  17  conference  abstracts  (A3 8-54) 

In  addition  we  are  currently  working  on  at  least  4  further  scientific  publications 
relevant  to  the  subject  area  of  this  grant  where  the  manuscripts  are  either  almost  ready 
for  submission  (041-42)  or  where  the  data  have  been  analyzed  and  we  are  currently 
writing  up  the  results  (043-44). 


1.2  Scientific  Workshops,  Conferences,  and  Seminars 

From  October  2010  until  today  I  was  invited  to  8  international  workshops  and 
conferences  including  (i)  an  ESF-EMBO-fimded  symposium  on  “Neurobiology  in 
Minibrains:  From  flies  to  robots,  and  back  again”,  Sant  Feliu  de  Guixols,  Spain, 
(2010),  (ii)  a  topical  conference  on  “Fly  Vision”  at  the  Howard  Hughes  Medical 
Institute,  Janelia  Farm  Campus,  Washington  DC,  USA,  (2011),  (iii)  the  4th  annual 
workshop  on  “Bio-inspired  control  design”  at  the  Air  Force  Research  Laboratory, 
Eglin,  FL,  USA,  (2011),  (iv)  a  cross-Atlantic  topical  meeting  on  “Bio-MA  V SOAR  ”  at 
Chilworth  Manor,  UK,  supported  by  the  Air  Force  Office  of  Scientific  Research,  USA 
&  Defense  Science  and  Technology  Laboratory,  MoD,  UK  (2001),  (v)  a  congress  on 
“Flow  sensing  in  air  and  water”,  at  the  University  of  Bonn,  Germany,  where  I 
presented  the  public  plenary  lecture  of  the  event  (2011),  (vi)  the  5th  annual  workshop 
on  “Bio-inspired  control  design”  at  the  Air  Force  Research  Laboratory,  Eglin,  FL, 
USA,  (2012),  (vii)  a  conference  on  “Bioactive  Amines”,  Freie  Universitat  Berlin, 
Germany,  where  the  talk  was  presented  by  Dr.  Kit  Longden  (2012),  and  (viii)  the  1st 
conference  on  Biomimetic  and  Biohybrid  Systems,  “Living  machines”,  Barcelona, 
Spain,  (2012).  I  was  also  invited  to  give  seminar  talks  at  (a)  the  University  of 
Tubingen,  Germany,  (2010),  (b)  the  University  of  Newcastle,  UK,  (2011),  and  (c)  the 
Heriot-Watt  University,  Edinburgh,  UK,  (2012). 
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Total  number  of  invitations  to  scientific  workshops,  conferences  and  seminars 
within  the  funding  period: 

•  17  invitations  to  present  at  international  scientific  conferences 

•  5  national  and  international  invitations  to  give  a  seminar  talk 

Next  year  I  have  been  invited  to  attend  4  international  conferences.  One  will  take 
place  in  London  where  I  will  act  as  local  co-organizer  (see  below)  and  another  one 
where  I  will  be  presenting  a  plenary  lecture.  Kit  Longden  was  invited  to  present  his 
work  on  state-dependent  processing  in  the  fly  motion  vision  pathway  at  two 
conferences  this  year.  He  was  presenting  at  a  topical  conference  on  bioactive  amines 
in  Berlin  (see  above)  and  at  the  ISN  meeting,  University  of  Maryland,  as  one  of  the 
speakers  in  a  dedicated  symposium  (see  below).  Kit  has  also  been  invited  to  give  two 
oral  presentations  at  conferences  next  year. 


1.3  Poster  and  Oral  Presentations  at  National  and  International  Conferences 

This  year  members  of  my  group  will  present  3  posters  at  the  International  Congress  of 
Neuroethology  (2012,  University  of  Maryland,  USA)  and  2  posters  at  the  European 
Microscopy  Congress  (2012,  Manchester,  UK).  At  both  conferences  one  of  our 
contributions  will  be  an  oral  presentation  (cf.  A50-54). 

From  2010-2011  we  presented  another  6  posters  at  international  meetings  (cf.  A44- 
49). 

Total  number  of  poster  and  oral  presentations  within  the  funding  period: 

•  17  conference  abstracts  (A38-54;  cf.  above) 


1.4  Funding 

Current  funding  secured  for  research  in  my  laboratory  (excluding  FA8655-09-1- 
3022): 

•  HFSP  research  collaboration,  PI,  with  three  other  Pis:  Drs.  Manos  Drakakis, 
Imperial  College,  Fabrizio  Gabbiani,  Baylor  College  of  Medicine,  TX,  USA, 
Martin  Egelhaaf,  Bielefeld  Universty,  Germany:  “Comparative  analysis  of 
RF-transmitted  neural  activity  on  flying  insects.  ”  [Non  cost  extension  until 
end  of  September  2012] 

•  AFOSR,  through  EOARD,  PI,  together  with  Dr  Sean  Humbert,  PI  University 
of  Maryland:  “The  relationship  between  visual  sensor  equipment  in  flying 
insects  and  their  flight  performance  -  a  ‘Neurobio-Engineering  ’  approach.  ” 

[Funding  for  work  in  my  lab  until  end  of  September  2013] 

•  Wellcome  Trust  research  grant,  PI:  “Integrated  reflex  control”  in  Manduca. 

[Funding  until  end  of  April  2014] 
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•  DSTL-sponsored  EPSRC  Industrial  CASE  PhD  Studentship,  PI:  “The 
significance  of  (image)  gaze  stabilization  -  Comparative  studies  on  gaze 
control  design  in  flying  insects  using  behavioural,  computational,  and 
electrophysiological  techniques.  ”  [Funding  until  end  of  March  2015 ] 

•  DSTL  National  PhD  Programme,  PI:  “The  significance  of  (image)  gaze 
stabilization  -  functional  characterization  of  the  neck  motor  system  using 
electrical  stimulation  of  identified  neck  muscles  in  blowflies.  ”  [ Funding  until 
end  of  September  2016 ] 


Grant  proposals  submitted: 

•  AFOSR/EOARD,  PI,  together  with  Dr  Graham  Taylor,  PI,  University  of 
Oxford,  UK,  and  Dr  Sean  Humbert,  PI,  University  of  Maryland:  “The  mode 
sensing  hypothesis  ”  [. Decision  pending ] 

•  Seventh  Framework  Programme  (FP7)  with  the  European  Commission.  Co-PI 
on  an  international  consortium  of  neuroscientists,  engineers,  theoreticians 
from  the  UK,  Germany,  and  Switzerland  on  “ ChemXplore  -  visually-guided 
exploration  of  chemical  landscapes.  ”  My  part  will  include  experimental  and 
modelling  work  on  a  multisensory  (vision-olfaction)  control  design  for 
autonomous  micro-air-vehicle  application.  [Invited  to  submit  full  proposal]. 

Grant  proposals  in  preparation: 

•  We  are  currently  preparing  the  full  FP7  ChemXplore  proposal  which  is  highly 
relevant  in  the  context  of  micro-air-vehicle,  guidance,  navigation  and  control. 
To  my  knowledge,  there  is  no  aerial  robotic  system,  so  far,  that  combines 
visual  and  olfactory  information  for  GNC. 


1.5  Group  size,  Collaborations,  and  Scientific  Impact 


Group  size: 

My  group  currently  consists  of  2  RAs  (postdocs)  and  4  postgraduate  students.  In 
addition  I  currently  supervise  4  MSc  project  student  and  4  UG  summer  internship 
students.  I  also  co-supervise  a  DSTL- funded  DPhil  student  together  with  Graham 
Taylor  at  the  University  of  Oxford.  Should  the  current  grant  application  with 
AFOSR/EOARD  be  successful  I  will  employ  another  RA.  I  am  also  looking  into 
funding  opportunities  for  more  PhD  students  as  two  of  my  current  PhD  students 
will  submit  their  theses  within  the  next  few  months. 


Collaborations: 
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At  Imperial  College  I  am  collaborating  with  several  groups  in  the  Department  of 
Bioengineering:  Dr  Martyn  Boutelle  (Biosensors),  Dr  Simon  Schultz  (Neural 
Coding),  Dr  Manos  Drakakis  (Low-power  VLSI  technology),  and  Dr  Reiko 
Tanaka  (Compound  Control).  To  study  the  functional  anatomy  of  insects  we  still 
collaborate  with  the  micro-CT  group  at  the  Natural  History  Museum,  London. 
More  recently,  we  have  also  started  to  collaborate  with  scientists  at  the  Paul 
Scherre  Institute,  Swiss  Light  Source,  Switzerland,  to  obtain  high-resolution  4D 
data  from  tethered  flying  flies  on  the  functional  organization  of  their  fight  and 
neck  motor  systems.  Collaboration  with  Prof  Simon  Laughlin  (Cambridge)  and 
Dr  Graham  Taylor  (Oxford),  have  been  maintained  and  will  be  intensified  by 
means  of  joint  grant  applications.  International  Collaborations  include  work  with 
Prof  Martin  Egelhaaf,  Neurobiology,  Bielefeld  University,  Germany;  Prof 
Fabrizio  Gabbiani,  Baylor  College  of  Medicine,  USA;  Prof  Sean  Humbert, 
Department  of  Aerospace  Engineering,  University  of  Maryland,  USA;  and  Mr 
Ric  Wehling  at  the  AF  Research  Laboratory,  Eglin,  US. 


Bibliometric  Data  and  Scientific  Impact: 

Bibliometrics: 


Nov  2008 

Aug  2012 

Number  of  publications* 

27 

46 

Number  of  citations** 

753 

1619 

h-factor** 

16 

23 

*  =  including  reviewed  conference  proceedings,  excluding  book  chapters 
**  =  according  to  “Harzing’s  Publish  or  Perish” 

Over  the  funding  period  (October  2008  until  end  of  March  2012)  there  has  been  a 
significant  increase  of  scientific  outputs  and  impact  regarding  the  number  of  both 
publications  and  citations. 


Scientific  Impact 

•  Refereeing  manuscripts  for  more  than  30  peer  review  journals,  including 

Nature,  Science,  and  Neuron. 

•  Refereeing  grant  applications  for  9  funding  agencies  including  RCUK,  NSF 
(USA),  and  AFOSR  (USA). 

•  Expert  Reviewer  for  European  Commission  FP7  collaborative  project 

•  Member  of  Academic  Editorial  Board  of  open  access  journal  PLoS  ONE 

•  Financial  Co-organizer  AFOSR/DSTL  “Zh'o  MA  V  SOAR ”  meeting,  Chilworth 
Manor,  201 1  and  UK  follow-up  meeting  at  Imperial  College,  London,  2012. 
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•  Member  of  Programme  Committee  of  Conference  on  Biomimetic  and 
Biohybrid  Systems  “Living  Machines”,  Barcelona,  2012 

•  Co-ordination  of  DSTL-supported  virtual  Centre  of  Excellence  on  Unmanned 
Autonomous  Systems  ( UAS  -  former  “MA  V”)  with  meetings  at  Imperial 
College,  Feb  2012,  and  at  the  University  of  Oxford,  Aug  2012 

•  Local  Co-organizer  of  Conference  on  Biomimetic  and  Biohybrid  Systems 
“Living  Machines”,  London,  2013 

•  Co-applicant/proposer  of  research  call  initiatives  in  the  area  of  sensing  and 
actuation  (ESF/NSF)  and  “Robot  Companions”,  Flagship  proposal  submitted 
to  the  European  Commission. 

In  parallel  with  the  increase  of  my  bibliometric  outputs,  I  have  also  taken  on 
more  responsibility  in  terms  of  (i)  fostering  visibility  and  promote  areas  relevant 
to  the  remits  of  AFRL/AFOSR  support  and  (ii)  contributing  to  a  high  standard  in 
conducting  scientific  research  by  expanding  my  commitment  to  reviewing  and 
editorial  work. 


2.  Report  on  current  projects: 

My  scientific  interest  focuses  on  the  neural  mechanisms  underlying  the  control  of 
behaviour.  In  my  lab  we  perform  comparative  studies  on  sensorimotor  control  in 
flying  insects  which  are  based  on  a  systems  neuroscience  approach.  We  aim  to 
discover  general  principles  and  species-specific  adaptations  regarding  the  relationship 
between  morphological  structures,  neural  information  processing  and  motor  control. 
Most  of  my  previous  research  focussed  on  visuomotor  control  in  blowflies  and  locust. 
To  identify  general  principles  of  sensorimotor  control  I  have  recently  included  several 
other  dipteran  flies  and  moths  ( Manduca  sexto)  as  experimental  model  systems. 
Together  these  model  systems  show  a  sufficiently  large  degree  of  diversity  to  extract 
specific  adaptations  and  common  functional  principles  in  terms  of  how  behaviour, 
morphology,  and  neuronal  processing  and  motor  control  are  linked  together. 

Over  the  funding  period  of  this  project,  I  have  been  developing  four  complementary 
research  priorities  which  are  meant  to  advance  our  understanding  of  the  neural  basis 
of  biological  control  design  and  its  translation  into  technical  applications.  These 
research  priorities  include: 

2.1.  Behavioural  performance  of  multisensory  motor  control  in  dipteran  flies 

2.2.  Biomechanics  of  motor  systems 

2.3.  Neural  mechanisms  underlying  multisensory  motor  control  in  dipteran  flies 

2.4.  Modelling  of  multisensory  motor  control  design 

In  the  following  I  will  report  progress  on  these  research  priorities  which  are  funded  by 
AFRL/AFOSR,  HFSP,  Wellcome  Trust,  DSTL  and  Imperial  College  resources,  or  are 
continuations/expansions  of  collaborations  based  on  previous  funding. 
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2.1.  Behavioural  performance  of  multisensory  motor  control  in  dipteran  flies 

In  previous  reports  I  emphasised  the  significance  of  gaze  stabilization  for  the  stunning 
aerodynamic  capabilities  of  dipteran  flies.  In  particular  bigger  species  where  inertial 
forces  have  a  significant  impact  on  the  animal’s  flight  dynamics  strongly  rely  on  gaze 
stabilization  to  (a)  reduce  motion  blur  and  to  (b)  establish  and  maintain  a  common 
frame  of  reference  relative  to  the  inertial  vector  for  head-based  sensor  systems  such  as 
antennae,  compound  eye  and  ocelli.  Because  the  neck  motor  system  of  an  insect 
initiates  compensatory  head  movements  whenever  the  body  changes  orientation  due 
to  external  disturbances  or  voluntary  trajectory  changes  -  gaze  stabilization  is 
intimately  linked  to  flight  stability  and  control. 

We  have  performed  quantitative  behavioural  experiments,  over  the  last  couple  of 
years,  using  a  linear  systems  analysis  approach  to  characterize  the  dynamic  responses 
of  the  fly  gaze  stabilization  system  within  a  control  engineering  framework  (037). 
One  of  the  most  important  properties  of  fly  gaze  and  flight  control  is  that  they  rely  on 
signals  from  a  wide  variety  of  sensor  systems  (024).  According  to  studies  by 
Hengstenberg  (e.g.  rev.:  1993)  in  the  velocity  domain,  signals  from  different  sensor 
systems  are  internally  scaled  and  are  then  linearly  combined  to  control  compensatory 
head  movements.  Earlier  anatomical  (Strausfeld  et  al.  1987)  and  electrophysiological 
(Wertz  et  al.  2007)  work  supports  the  idea  that  those  signals  are  simply  added  in 
descending  neurons  by  employing  electrical  synapses.  On  the  other  hand,  there  is 
evidence  that  some  neck  motor  neurons  involved  in  gaze  stabilization  have  highly 
non-linear  signal  integration  properties  in  that  they  only  generate  action  potentials 
when  receiving  input  from  at  least  two  different  sensors  (e.g.  030). 

To  guide  our  studies  on  the  neural  mechanisms  underlying  multisensory  gaze  and 
flight  stabilization,  we  performed  a  couple  of  behavioural  experiments  on 
compensatory  head  roll  in  Calliphora  where  we  systematically  manipulated  the 
combination  of  sensor  systems  contributing  to  the  response.  Head  roll  was  monitored 
using  a  high-speed  video  camera  while  the  animal  was  subjected  to  sinusoidal 
stimulation  of  various  sensors  to  obtain  the  respective  frequency  response.  From  the 
frequency  responses  under  closed-loop  conditions  we  derived  transfer  functions 
which,  in  turn,  could  be  used  to  set  up  a  simulation  platform  for  fly  gaze  stabilization 
(see  also  below).  We  chose  a  linear  systems  analysis  approach.  This  was  justified  by 
two-tone  modulation  experiments,  where  the  sum  of  two  stimulation  frequencies  only 
produced  small  contributions  of  higher  harmonics  in  the  behavioural  response  - 
which  suggests  a  mostly  linear  system  (037). 

There  are  two  sets  of  fundamental  questions  regarding  multisensory  reflex  control: 
Firstly,  what  is  the  design  of  the  control  architecture  underlying  gaze  and  flight 
stabilization?  More  specifically,  how  are  visual,  mechanosensory  and 
proprioreceptive  signals  -  measured  by  head-  and  thorax-based  sensors  (024)  - 
combined  in  the  feedforward  and/or  feedback  sense  to  provide  reasonably  robust 
motor  commands?  Secondly,  how  are  the  responses  -  each  of  which  incurs  a 
pathway-  specific  response  delay  -  amplified/attenuated  to  enable  a  sufficiently  large 
bandwidth  without  compromising  stable  control?  In  this  context  it  should  be 
particularly  interesting  to  review  the  motion  vision  pathway  supplied  by  the 
compound  eye.  In  light-adapted  flies,  motion  vision  is  primarily  (70%)  based  on  a 
spatio-temporal  correlation  of  light  levels  measured  in  adjacent  ommatidia  within  the 


Distribution  A:  Approved  for  public  release;  distribution  is  unlimited. 


hexagonal  eye  lattice.  The  spatial  sampling  base  determined  by  the  interommatidial 
angle  between  neighbouring  facets  in  combination  with  the  time  constant  in  the 
elementary  movement  detect  (rev.:  024)  results  in  a  optimal  response  of  the  system  to 
pattern  motion  in  the  range  of  hundreds  of  degrees  per  second  (rev.:  e.g.  Hausen 
1993).  Why  does  the  system  not  employ  motion  detectors  with  larger  sampling  bases 
to  shift  its  maximum  sensitivity  to  higher  angular  velocities  by,  for  instance,  including 
next-but  neighbouring  ommatidia?  A  tentative  answer  may  come  from  the  fact  that 
the  response  delay  in  the  motion  vision  pathway  is  rather  long  due  to  the  extra  time 
required  to  compute  directional  motion.  Just  increasing  gain  and  bandwidth  but 
leaving  the  response  delay  unchanged  potentially  bears  the  risk  of  instabilities  in  a 
closed-loop  feedback  system. 

Our  previous  and  recent  experiments  on  Calliphora  compensatory  head  roll  addressed 
these  two  sets  of  questions  focusing  on  the  architecture  and  stability  of  the  control 
system  (037,  041). 

We  performed  experiments  where  the  compensatory  head  roll  was  measured  in 
animals  whose  thorax  was  oscillated  by  +/-  40  degrees  and  at  different  frequencies  for 
two  conditions:  Cl  =  stimulating  the  compound  eye  motion  vision  pathway  and  the 
halteres,  and  C2  =  only  stimulating  the  compound  eye  motion  vision  pathway.  The 
results  suggested  that  signals  obtained  by  these  two  major  sensor  systems  are 
combined  according  to  an  architecture  that  is  known  in  engineering  as  two  degrees  of 
freedom  controller.  One  control  degree  is  provided  by  the  halters  which  mediate  a 
feedforward  response  that  results  in  an  initial  compensatory  head  roll.  This  fast 
response  component  effectively  reduces  the  remaining  retinal  slip  speed  to  a  velocity 
range  the  motion  vision  pathway  is  most  sensitive  to.  The  second  control  degree 
corresponds  to  negative  feedback  provided  by  the  motion  vision  pathway  in  an 
attempt  to  compensate  for  the  remaining  retinal  slip  speed.  The  fast  response 
component  initiated  by  the  halteres,  in  a  way,  linearizes  the  system  as  it  shifts  the 
distribution  of  retinal  slip  speeds  into  the  operating  range  of  the  motion  vision 
pathway  (037,  041).  The  methodology  used  and  results  obtained  are  presented  in 
Figures  1-3  (Schwyn  et  al.,  in  prep,  041). 
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a) 


c) 


Figure  1:  Experimental  setup  (a)  and  analysis  (b)  of  compensatory  head  roll  (HR)  during 
externally  forced  thorax  rotations  (TR).  The  performance  of  the  neck  motor  system  is 
reflected  by  the  difference  HR-TR.  Perfect  compensation  for  TR  would  result  in  HR  =  0.  (c) 
TR  and  HR  are  shown  as  a  function  of  time,  (from  Schwyn  et  al.  in  prep,  041). 
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in .  Figure  2:  (a)  Bode  plot  of 

frequency  responses  of 
Calliphora  compensatory 
head  roll  for  thorax 
oscillation  amplitudes  of 
+/-  40  degrees.  Cl  and 
C2  refer  to  stimulation  of 
the  motion  vision 
pathway  with  and  without 
the  halters  being  intact, 
respectively.  Note  that 
without  halters  the  roll 
off  frequency  is  shifted  to 
lower  stimulation 

frequencies.  N=4 

different  flies,  (b)  Two 
degrees  of  freedom 
controller.  The  forward 
component  is  provided  by 
the  thorax-based  halteres 
which  initiate  a  fast 
compensatory  head  roll. 
The  remaining  retinal 
slip  speed,  i.e.  the 
relative  motion  between 
the  compound  eyes  and 
the  fixed  visual 

environment,  is  then  sensed  by  the  motion  vision  pathway  which  generates  the  2nd  degree  of 
freedom  of  the  controller  mediating  a  negative  feedback  signal.  (From  Schwyn  et  al.,  in 
preparation,  041). 
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From  the  results  shown  in  Fig.  2,  the  open-loop  pathway  transfer  functions  can  be 
calculated  (Fig.  3  c),  which  in  turn  are  used  to  assess  the  stability  margins  of  the 
motion  vision  pathway  in  combination  with  the  neck  motor  system.  The  rational  of 
focusing  on  the  feedback  degree  of  freedom  of  the  control  architecture  shown  in  Fig. 
2b  is  that  the  slowest  component  on  its  own  is  required  to  perform  in  a  stable  way  for 
the  entire  controller  to  be  stable. 


d) 


Frequency  /  Hz 


Fig  3:  (c)  Open-loop 

transfer  function  for  the 
feedback  (green)  and 
feedforward  (blue) 

components  of  the  2  DoF 
controller,  respectively, 
(d)  Stability  margins  for 
the  open-loop  pathway 
transfer  function  involving 
the  motion  vision  pathway. 
Note  the  surprisingly  high 
gain  (upper  plot,  GM)  and 
phase  (lower  plot,  PM) 
margins,  respectively. 
Green  circles  give  the 
mean  values  obtained 
from  four  animals  with 
magenta  lines  indicating 
the  standard  deviation, 
(from  Schwyn  et  al.,  in 
prep,  041). 


Surprisingly,  our  analysis  of  the  transfer  function  including  the  motion  vision  pathway 
indicated  unexpectedly  high  stability  margins.  The  gain  margin  (Fig.  3d,  GM)  at  the 
180  degrees  phase  reversal  is  still  20  dB,  and  the  phase  margin  at  the  zero-crossing  in 
the  gain  plot  amounts  to  130  degrees  (Fig.  3d,  PM). 

Given  these  values,  the  motion  vision  pathway  seems  to  be  totally  stable  and  could 
even  accommodate  longer  fixed  response  delays  without  risking  instable  performance. 
Why  is  there  so  much  leeway  build  into  the  controller?  Our  interpretation  is  that,  by 
implementing  extra  high  stability  margins,  the  control  system  can  cope  with  a  large 
amount  of  variability  -  or  noise  -  inherent  to  each  of  the  individual  sensory  pathways. 
Noise  distributions  in  different  sensor  systems  may  have  different  shapes,  which 
could  mean  that  even  if  the  sources  are  independent  they  may  not  cancel  out  each 
other.  Also,  expansive  non-linear  processes  when  combining  signals  from  different 
modalities  could  potentially  boost  the  feedback  signals  beyond  a  magnitude  that  is 
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tolerable  for  stable  control.  Further  studies,  specifically  aiming  to  reveal  potential 
non-linearities  in  multisensory  integration  are  on  its  way  (See  next  paragraph). 

As  mentioned  above,  signals  from  different  sensors  may  be  combined  in  a  non-linear 
way.  One  example  would  be  the  gating  effect  in  neck  motor  neurons  observed  during 
compound  eye  and  haltere  stimulation.  An  exciting  question  to  ask  is:  given  the  non¬ 
linear  processing  steps  in  the  motion  vision  pathway  and  other  neural  processing 
stages  known  to  affect  sensory  integration  (028,  032),  how  does  the  system  manage 
to  produce  an  overall  fairly  linear  behavioural  output?  Obviously,  for  each  non-linear 
process  there  should  be  another  non-linear  process  that  roughly  compensates  for  it. 

We  have  now  started  to  perform  behavioural  experiments  to  study  the  combination  of 
specific  sensor  systems,  for  instance  the  interaction  between  the  compound  eye  and 
the  ocellar  pathways  -  a  project  highly  relevant  in  the  context  of  the  joint  research 
grant  (FA8655-09- 1-3083)  together  with  Dr  Sean  Humbert,  UMD.  Previous 
electrophysiological  work  has  shown  that  the  impact  of  the  ocellar  system  on  the 
motion  vision  pathway  can  be  observed  at  the  level  of  lobula  plate  tangential  cells 
(020,  036)  -  probably  due  to  electrical  synapses  connecting  to  descending  neurons 
which  integrate  signals  from  several  sensory  systems. 

Our  experiments  were  performed  on  tethered  flying  animals  in  a  static  position.  To 
stimulate  the  motion  vision  and  ocellar  systems  we  oscillated  a  dark  hemisphere  in  the 
ventral  visual  field  at  different  frequencies  and  amplitudes.  The  stimuli  mimicked  a 
rotation  of  the  animal  round  the  roll  axis.  Two  conditions  were  tested:  compound  eye 
+  ocellar  and  only  compound  eye  stimulation  (painting  over  the  ocelli),  respectively. 


the  phase  delay  for  responses  to  frequencies  above  3  Hz.  Mean  values  ofN=7  flies  +/-  SEM. 
Schwyn  et  al.,  work  in  progress. 


Interestingly,  the  gain  did  not  depend  on  whether  or  not  the  ocelli  contributed  to  the 
response  (Fig  4).  The  phase,  however,  was  advanced,  when  the  ocelli  were  stimulated 
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in  addition  to  the  compound  eyes.  This  result,  compatible  with  previous 
electrophysiological  studies  (036),  indicates  a  non-linear  integration  of  compound 
eye  and  ocellar  signals  at  the  descending  neuron  level.  As  pointed  out  in  previous 
publications  (e.g.  036,  Elzinga  et  al.  2011)  closed-loop  feedback  control  in  biological 
systems  heavily  relies  on  short  response  delays  to  prevent  instabilities.  The 
behavioural  data  (Fig.  4)  show  that  the  combination  of  compound  eye  and  ocellar 
signals  mainly  achieves  a  shorter  response  delay  without  increasing  the  gain.  This 
finding  could  be  interpreted  as  evidence  that  reducing  response  delays  in  pathways 
mediating  negative  feedback  signals  might  indeed  be  a  fundamental  aspect  of 
multisensory  integration  in  biological  control  systems. 

A  potential  mechanistic  model  which  may  explain  those  results  would  include  a  static 
threshold  non-linearity  as  presented  in  figure  5  (A50).  Including  another  level  of 
modelling  -  here  mechanistic  in  addition  to  phenomenological  -  will  be  required  to 
incorporate  neural  mechanisms  of  multisensory  signal  integration  as  a  complementary 
approach  to  a  systems  analysis/identification  adapted  from  engineering. 


compound 

eyes 


input 


ocelli 


Figure  5:  mechanistic  model  of  non-linear  combination  of  compound  eye  and  ocellar  signals 
at  the  level  of  a  descending  neuron.  This  simplified  model  approximates  the  transfer  functions 
of  the  compound  eye-mediated  motion  vision  pathway  and  the  ocellar  pathway  by  1st  order 
low-pass  and  high-pass  fdters,  respectively,  in  combination  with  individual  pathway  gains. 
The  output  of  both  visual  systems  is  integrated  by  a  descending  neuron  that  converts 
membrane  potential  changes  into  trains  of  action  potential  through  a  static  threshold  non¬ 
linearity.  The  model  captures  the  experimental  result  that  ocellar  signals  do  not  increase  the 
overall  gain  along  the  combined  pathway  but  reduce  the  phase  shift  of  the  response,  (work  in 
progress). 


In  conclusion,  over  the  funding  period  of  this  grant  we  have  performed  a  number  of 
behavioural  studies  to  evaluate  gaze  stabilization  behaviour  in  Calliphora  within  a 
control  engineering  framework.  Using  a  linear  systems  approach  to  start  with,  we 
were  able  to  propose  a  controller  design  (two  degrees  of  freedom)  that  combines 
mechanosensory  feedforward  with  visual  feedback  signals.  We  also  demonstrated  the 
presence  of  non-linear  processes,  probably  at  the  level  of  the  descending  neurons,  in 
the  context  of  compound  eye  and  ocellar  signal  integration. 

Our  results  seem  to  be  in  contradiction  with  electrophysiological  studies  suggesting  a 
linear  combination  of  compound  eye  and  ocellar  signals.  However,  the  nature  of  a 
threshold  non-linearity  as  proposed  in  the  block  diagram  in  Fig  5  does  not  exclude  a 
linear  combination  of  inputs  once  the  threshold  potential  for  triggering  action 
potentials  is  reached.  Beyond  the  threshold  potential,  inputs  may  result  in  a  linear 
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increase  of  the  descending  neuron’s  spike  rate  with  increasing  signal  amplitudes 
mediated  by  the  compound  and  ocellar  systems.  Similarly,  sub-threshold  inputs  may 
indeed  add  up  linearly  as  demonstrated  by  Haag  et  al.  (2007)  in  recordings  from 
ocellar  L-neurons,  LPTCs  and  descending  neurons.  Sub-threshold  activity  changes, 
however,  would  not  result  in  any  behavioural  output  as  the  latter  requires  the 
generation  of  action  potentials  to  induce  muscle  contraction  via  motor  neurons. 
Further  experiments  will  be  needed  to  pinpoint  the  exact  mechanism  underlying 
ocellar  and  compound  eye  signal  integration. 
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Figure  6:  Roll  gaze  stabilization  in  the  robberfly  (Dysmachus  spec.)  at  different  thorax 
oscillation  frequencies.  Green  and  blue  traces  show  the  stimulus  and  compensatory  head  roll 
movements  over  time,  respectively.  Note  that  in  this  case  the  blue  trace  indicates  the  action  of 
the  neck  motor  system.  Bottom  right:  Dysmachus  Spec.  Source:  Geller-Grimm  (A53, 
Hardcastle  et  al.  work  in  progress.) 


We  have  also  begun  to  perform  gaze  stabilization  experiments  in  dipteran  species 
other  than  Calliphora.  First  experiments  on  a  UK  robberfly  species  ( Dysmachus  spec, 
Fig.  6)  and  on  hoverflies  (. Eristalis  spec )  suggest  that  both  these  animals  also  perform 
compensatory  head  roll  movements  when  subjected  to  thorax  rotations. 


Two  exciting  results  are:  (a)  robberflies  do  show  head  roll  responses  independent  on 
whether  or  not  they  are  flying  -  i.e.  wing  beat  generation  is  not  a  necessary  condition 
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for  compensatory  head  roll  to  occur  as  it  is  for  Calliphora  and  (b)  hoverflies,  which 
were  thought  not  to  perform  compensatory  head  roll  responses  at  all  (Collett  and 
Land,  van  Hateren  lab),  did  in  fact  show  head  roll  responses  under  the  experimental 
conditions  we  applied,  at  least  in  some  cases.  It  is  likely  that  previous  work  on 
hoverfly  head  roll  was  suffering  from  a  sampling  bias,  i.e.  many  hoverflies  do  not 
show  head  roll  but  some  actually  do.  A  bimodal  distribution  of  animals,  part  of  which 
did  perform  head  movements  while  the  other  part  did  show  a  near-zero  gain  had  been 
reported  for  blowflies  by  Rosner  et  al.  (2009). 


2.2.  Biomechanics  of  motor  systems 

Up  to  a  certain  degree  -  systems  analysis  approaches  in  a  linear  control  engineering 
framework  have  been  successful  to  provide  phenomenological  models  of  the  overall 
performance  in  stabilization  behaviours.  However,  they  do  not  allow  us  to  directly 
specify  the  transfer  functions  of  the  relevant  motor  systems.  Whether  gaze 
stabilization  or  flight  control  are  considered,  the  behavioural  outputs  have  to  be 
interpreted  as  pathway  transfer  functions  which  reflect  both  the  properties  of  the 
sensor  systems  in  series  with  the  properties  of  the  respective  motor  system.  When 
deriving  transfer  functions  for  the  motion  vision  and  the  haltere  pathways  from 
behavioural  data  in  an  earlier  study  (028)  we  were  able  to  avoid  the  problem  of  not 
having  the  full  specifications  of  the  neck  motor  system  by  considering  ratios  of  both 
pathway  transfer  functions  to  cancel  out  neck  motor  system  contributions. 

An  alternative  approach  would  be  to  derive  the  dynamic  properties  of  the  neck  and 
flight  motor  systems  in  a  more  direct  way  by  characterizing  their  functional 
anatomical  organization.  To  this  end  we  have  engaged  on  studies  using  x-ray  based 
techniques  such  as  micro-Computer  Tomography  (p-CT)  to  obtain  3-dimensional 
reconstructions  of  fly  motor  systems  (045,  A53).  The  results  obtained  so  far  in  the 
course  of  collaborations  with  the  Natural  History  Museum  London  proofed  successful 
in  that  we  were  able  to  achieve  sufficient  tissue  contrast  and  spatial  resolution  to 
digitize  and  visualizes  parts  of  both  the  neck  and  the  flight  motor  system.  Based  on 
these  data  the  next  step  will  be  to  create  3 -dimensional  models  of  the  motor  systems 
by  means  of  rapid  prototyping.  Once  such  models  are  available  we  should  be  in  the 
position  to  reconstruct  pulling  planes  of  direct  and  indirect  muscles  which  are 
involved  in  the  actuation  of  head  movement  and  the  control  of  the  flight  motor.  Figure 
7  illustrates  the  level  of  resolution  achievable  with  static  p-CT  methodology. 

Over  the  last  two  years,  we  have  accumulated  a  substantial  data  base  which  not  only 
provides  valuable  information  for  a  reconstruction  of  the  neck  and  flight  motor  system 
but  also  to  quantify  other  anatomical  and  brain  structures  including  the  volumes  of  the 
optic  lobes  in  male  and  female  Calliphora  (A48). 

Another  exceptionally  interesting  aspect  of  a  3-dimensional  anatomical  representation 
based  on  p-CT  is  based  on  identifying  homolog  structures  across  different  dipteran  fly 
species.  Once  those  structures  have  been  identified  in  a  sufficient  number  of  animals, 
principle  component  analysis  enables  us  to  extract  those  parameters  which  best 
describe  the  anatomical  differences  between  species.  For  instance,  robberflies  and 
blowflies  show  significant  differences  regarding  the  angular  range  within  which  the 
animals  are  able  to  move  their  head  relative  to  the  body.  They  also  show  marked 
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differences  with  respect  to  the  size  and  power  of  their  flight  motor  where  robberflies 
are  capable  of  taking  on  payload  (prey)  significantly  higher  than  their  own  body 
weight.  Knowing  the  principle  components  associated  with  interspecific  anatomical 
differences  of  the  flight  motor  would  allow  us  to  correlate  them  with  entirely 
independent  properties  or  features  such  as  the  number  of  lobula  plate  tangential  cells, 
flight  envelop,  or  observed  modes  of  motion.  Such  analysis  would  offer  an  incredibly 
powerful  tool  to  study  the  evolution  of  species-specific  adaptations  in  senorimotor 
ecology  -  which  arguable  is  one  of  the  keys  to  understanding  general  principles  of  the 
relationship  between  sensing  and  actuation  in  flying  insects. 


Figure  7:  Micro-CT  image  of  the  head  and  parts  of 
the  thorax  of  Calliphora,  ventral  view.  The  figure 
shows  a  horizontal  section  through  the  3-dimensional 
structure  of  the  animal’s  head  and  thorax  which 
allows  us  to  choose  the  image  plane  as  required  for 
the  rendering  of  any  arbitrary  perspective  on  various 
anatomical  details.  Bright  and  dark  structures  in  the 
thorax  (lower  part)  and  in  the  neck  region  represent 
muscles  and  cuticle,  respectively.  The  spatial 
resolution  is  in  the  range  of  5  pm.  In  the  neck  motor 
region,  a  pair  of  direct  muscles  and  their  attachment 
points  at  specialized  cuticular  structures,  condyles, 
are  visible.  In  the  thorax  the  dorsoventral  (cross- 
section)  and  parts  of  the  dorsal  longitudinal  (partial 
horizontal  section)  power  muscles  can  be  seen,  (work 
in  progress,  045). 


Although  p  -CT  data  provide  essential  3-dimensional  data  of  functional  anatomy, 
because  the  data  are  obtained  in  dead  specimens,  inferences  about  the  dynamic 
properties  of  the  motor  systems  are  difficult  to  make.  To  overcome  these  limitations 
we  are  now  including  more  powerful  x-ray-based  techniques,  i.e.  in  vivo  synchrotron 
measurements.  We  were  awarded  measuring  time  for  two  projects  conducted  at  the 
Swiss  Light  Source  on  the  TOMCAT  beam  line  at  the  Paul  Scherre  Institute  (PSI)  to 
perform  ultra  fast  p-CT  scans  in  tethered  flying  flies.  The  experiments  and  the 
analysis  of  the  data  is  a  joint  effort  of  Dr  Graham  Taylor’s  group,  University  of 
Oxford,  the  local  scientists  at  the  PSI  and  my  group  (PI).  Our  first  results  are 
fascinating  in  that  they  allow  us  (a)  to  segment  different  tissue  types  based  on  phase 
contrast  rather  than  on  issue  absorption  (b)  render  different  aspects  of  steering  muscle 
activity  related  to  the  control  of  the  fly’s  wing  beat,  (c)  to  quantify  volume  and  length 
changes  of  fly  power,  steering,  and  neck  muscles,  (d)  to  quantify  non-linear  cuticular 
deformations  which  may  contribute  to  efficient  lift  production  due  to  energy  storage 
in  the  cuticle,  and  finally  (e)  the  periodic  volume  changes  of  the  tracheal  system  that 
provides  the  oxygen  supply  to  the  power  muscles  (A52). 

Figure  8  shows  a  pseudo  3-dimensional  image  from  one  of  our  4-dimensional  data 
sets,  where  different  anatomical  structures  are  marked  by  different  colour  shades.  The 
availability  of  data  regarding  the  static  anatomical  organization  of  motor  systems  in 
combination  with  dynamic  synchrotron  measurements  will  be  instrumental  for  a 
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meaningful  reconstruction  of  the  biomechanics  involved  in  flight  and  gaze 
stabilization.  Also,  the  3-dimensional  representation  of  the  neck  motor  system  will  be 
instrumental  in  guiding  the  placement  of  electrodes  used  for  electrical  stimulation  of 
direct  neck  muscles  in  the  context  of  a  DSTL-funded  PhD  project  on  the  dynamics  of 
the  neck-motor  system. 


Figure  8:  Segmented  and  colour- 
labelled  anatomical  structures 
obtained  from  a  4-dimensional 
data  set  on  Calliphora.  Lateral 
view  of  the  thorax  (left)  - 
featuring  the  prominent  dorso- 
ventral  (pink,  violet)  and  dorsal 
longitudinal  (green)  muscles  of 
the  flight  motor  -  and  the  head 
(right)  which  are  connected  by 
the  neck  motor  system.  Note  that 
the  effective  spatial  resolution  is 
sufficiently  high  enough  to 
retrieve  the  articulations  and 
muscles  in  both  the  neck  and  in 
the  flight  motor  system.  (Schwyn 
et  al.  -  work  in  progress). 


There  are  currently  strong  joint  activities  between  Graham  Taylor’s  and  my  group  to 
optimize  segmentation  and  rendering  procedures  of  4-dimensional  data  sets  based  on 
commercially  available  software  packages.  Ideally,  we  would  engage  on 
collaborations  with  companies  providing  CT  scanners  and  associated  software 
solutions  to  advance  the  capability  of  existing  products.  4-dimensional  data  sets  -  in 
particular  when  obtained  from  comparatively  small  insects  -  still  present  a  significant 
challenge  for  tissue  segmentation. 

Nevertheless,  I  should  expect  that  due  to  our  integrated  approach  to  understand  the 
control  design  of  the  neck  motor  system  we  should  be  able  to  propose  a 
comprehensive  model  of  an  image  stabilization  system  that  is  truly  bio-inspired 
within  the  next  3  years. 


2.3.  Neural  mechanisms  underlying  multisensory  motor  control  in  dipteran  flies 

Behavioural  studies  are  key  to  assess  the  performance  limits  of  a  system  under  study 
in  terms  of  gaze  and  flight  control.  The  resulting  qualitative  and  quantitative 
descriptions  guide  our  studies  on  the  neural  mechanisms  enabling  stabilization 
reflexes.  Together  with  our  data  on  functional  anatomy,  the  behavioural  results 
provide  a  framework  that  constrains  the  way  in  which  the  nervous  system  may 
achieve  tasks  related  to  motor  control. 
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One  of  the  major  challenges  the  nervous  system  masters  is  the  transformation  of 
signals  obtained  in  local  sensory  coordinate  into  signals  that  can  be  used  to  control 
movements  in  coordinate  systems  essentially  defined  by  the  pulling  planes  of 
muscles.  Regarding  the  motion  vision  pathway  in  dipteran  flies  -  the  blowfly 
Calliphora  has  been  particularly  well  studied  in  terms  of  neuronal  mechanisms  (033). 
In  the  motion  vision  pathway,  noisy  local  signals  indicating  retinal  motion  along  the 
hexagonal  ommatidial  lattice  of  the  eye  are  selectively  integrated  at  the  lobula  plate 
tangential  cells  (LPTCs,  rev.:  024).  Formally,  this  selective  integration  of  local 
motion  signals  in  retinal  coordinates  results  in  LPTC  outputs  signalling  specific  self- 
motion  parameters  such  as  rotation  and  translation  components  by  processing  optic 
flow  information  (loc.  cit,  01,  03,  033).  LPTCs  are  connected  directly  and  indirectly 
-  via  descending  neurons  -  to  the  neck  and  flight  motor  systems.  In  terms  of  their 
preferred  self-motion  components  it  has  been  proposed  that  LPTCs  specifically  sense 
optic  flow  fields  that  are  induced  by  the  natural  modes  of  motion  of  a  flying  insect 
which  are  related  to  the  animal’s  aerodynamic  properties  (024,  R4). 

Such  a  modal  coordinate  system  set  up  by  the  LPTCs  would  serve  as  the  frame  of 
reference  regarding  the  species-specific  sensorimotor  transformations.  Information 
obtained  from  other  sensors  indicating  state  changes,  for  instance  the  ocelli,  could 
then  be  used  to  speed  up  the  response  within  the  inherently  slow  motion  vision 
pathway  (034,  see  above).  This  way  the  spatial  resolution  required  to  sense  specific 
modes  of  motion  is  established  by  choosing  and  integrating  those  local  directional 
inputs  which  match  the  optic  flow  vectors  generated  during  a  particular  mode  of 
motion.  As  the  compound  eye  in  flying  insects  have  an  enormous  number  -  several 
hundreds  in  Drosophila,  several  thousands  in  Calliphora  -  the  pool  of  local  inputs  to 
choose  from  potentially  allows  flying  insects  to  very  precisely  set  up  a  modal 
coordinate  system  in  the  motion  vision  pathway  that  satisfies  its  aerodynamic  control 
requirements.  The  ocelli,  on  the  other  hand,  provide  only  very  crude  spatial 
information  about  any  state  changes,  but  are  way  faster  than  the  motion  vision 
pathway  (020,  034,  028,  29). 

The  overall  performance  of  stabilization  reflexes  is  even  further  increased  regarding 
the  bandwidth  the  systems  are  able  to  cope  with,  by  including  fast  mechanosensory 
signals,  for  instance,  from  the  halteres  (rev.:  024).  As  we  know  from  both 
behavioural  studies  (037,  rev.:  024)  and  electrophysiology  (030)  the  halters 
contribute  to  gaze  and  flight  stabilization.  Recent  work  indicates,  though,  that  for  gaze 
stabilization  halter  signals  are  used  in  a  feed  forward  way  to  reduce  retinal  slip  speeds 
into  a  dynamic  range  the  motion  vision  pathway  can  operate  in  (037,  041).  For  flight 
control,  however,  the  signals  from  the  halteres  provide  immediate  negative  feedback 
to  the  flight  motor  (Elzinga  et  al.  2011). 

One  of  the  major  efforts  in  my  lab  concerns  the  characterization  of  response 
properties  in  optic  flow  processing  LPTCs.  This  is  because: 

o  LPTCs  assume  a  significant  role  in  the  sensorimotor  transformation.  Any  non¬ 
linear  processes  along  the  pathway  involving  motion  vision  may  only  be 
identified  in  terms  of  neuronal  activity  measurements  but  not  necessary  at  the 
behavioural  level. 
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o  The  response  properties  of  some  LPTCs  directly  reflect  multisensory 

integration  as  they  are  electrically  connected  to  descending  neurons  which  also 
receive  input  from  other  modalities. 

o  The  specific  receptive  field  organization  of  LPTCs  allows  us  to  derive  the  self- 
motion  components  different  species  of  flying  insects  have  evolved  to  detect 
and  most  likely  to  control. 

o  LPTCs  are  an  ideal  stage  in  the  sensorimotor  pathway  to  investigate  state- 
dependent  visual  information  processing.  This  topic  is  of  major  interest  when 
power  consumption  becomes  in  issue  due  to  limitation  in  payload  on  small 
terrestrial  and  aerial  autonomous  systems, 
o  LPTCs  also  are  likely  to  be  a  key  stage  regarding  the  integration  of  inner-loop 
stabilization  reflexes  and  outer-loop  tasks  related  to  navigation  and  distance  as 
well  as  proximity  estimation. 


LED-basecl  stimulation  device  to  map  the  receptive  field_  of  optic  flaw  processing 

interneurons  (LPTCs}  in  fly i ms  insects. 

In  previous  years  we  used  fast  CRT  displays  to  determine  general  visual  response 
properties  of  LPTCs  and  neck  motor  neurons  (e.g.  031,  036,  026,  030).  To  assess 
the  distribution  local  preferred  directions  and  motion  sensitivities  within  the  receptive 
field  of  LPTCs  we  mainly  applied  a  dedicated  electro-mechanical  stimulation  device 
optimized  to  speed  up  experimental  protocols  (02).  To  gain  more  flexibility  in  terms 
of  visual  stimulation  we  recently  re-designed  the  original  stimulation  device  by 
replacing  the  mechanical  stimulus,  a  black  disk  moving  on  a  circular  path,  with  LED 
displays.  Figure  9  shows  the  completed  device. 

Figure  9:  The  re-designed  stimulation  device 
consist  of  a  meridian  metal  frame  equipped  with 
6  LED  panels  mounted  at  +-  75,  45  and  15 
degrees  elevation.  The  spacing  of  the  LEDs  and 
the  rate  at  which  their  luminance  values  are 
refreshed  are  adapted  to  the  properties  of  the 
visual  system  under  moderate  light  levels.  The  fly 
in  positioned  in  the  centre  of  the  meridian  frame 
which  can  be  moved  around  the  animal  to  cover 
an  overall  azimuthal  range  of  about  315  degrees. 
Positioning  of  the  recording  electrode  is 
achieved  by  a  3  DoF  electronic  micro¬ 
manipulator  under  visual  magnification  using  a 
stereo  microscope.  (photograph:  Ben 

Hardcastle). 
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The  device  is  controlled  by  a  lab-view  interface  and  allows  us  to  deliver  visual  motion 
stimuli  to  determine  both  the  local  directional  tuning  of  LPTCs  and  their  dynamic 
response  properties.  It  has  been  successfully  tested  in  experiments  on  blowflies  so  far, 
but  will  also  be  instrumental  to  map  the  receptive  fields  of  other  dipteran  flies  such  as 
robberflies,  horseflies  and  hoverflies. 


Receptive  field_  organization  of  optic  flaw  processing  interneurons  in  various  dipteran 
ftp  species. 

Current  research  in  collaboration  with  Dr  Graham  Taylor,  Oxford,  and  Dr  Sean 
Humbert,  UMD,  has  made  considerable  progress  in  support  of  the  mode  sensing 
hypothesis  (e.g.  R4).  Free  flight  experiments  in  Sean  Humbert’s  lab  have  successfully 
been  carried  out  to  quantify  the  modes  of  motion  in  Calliphora.  We  are  currently 
working  on  a  scientific  publication  that  relates  the  modes  of  motion  to  the  directional 
templates  of  blowfly  LPTCs  to  establish  the  relationship  between  sensory  and  motor 
control  coordinate  systems  (044).  This  work  benefits  from  a  joint  AFOSR  research 
grant  to  Sean  Humbert  and  myself,  where  we  focus  on  the  interactions  between  the 
motion  vision  pathway  and  the  ocelli  (see  above)  and  will  be  reported  in  detail  in  a 
separate  paper. 

To  gain  further  support  for  the  mode  sensing  hypothesis  and  to  facilitate  the 
translation  of  biological  control  design  principles  into  enabling  technology  we  have 
applied  for  additional  funding  with  AFOSR  (decision  pending).  In  order  to  show  that 
the  mode  sensing  hypothesis  has  general  implication  for  biological  control  we  will 
have  to  apply  a  comparative  approach.  Across  several  dipteran  flies,  orthopterans,  and 
potentially  Lepidoptera  we  will  characterize  the  receptive  fields  of  optic  flow 
processing  interneurons  and  the  animals’  flight  dynamics.  These  species  cover  a 
sufficiently  broad  spectrum  of  flight  behaviours/aerodynamic  properties  to  test 
whether  the  alignment  of  sensory  and  motor  coordinate  systems  is  a  general  principle 
amongst  flying  insects. 

In  my  lab  we  have  started  recording  from  optic  flow  processing  interneurons  in 
robberflies,  horseflies  and  hoverflies.  Several  singular  recordings  from  US  robberfly 
species  such  as  Laphria  and  Diogmites  -  caught  by/or  in  collaboration  with  Ric 
Wehling,  AFRL,  Eglin  -  have  shown  that  spiking  LPTC  in  robberflies  have  response 
properties  similar  to  those  found  in  the  blowfly.  Although  we  have  not  yet  performed 
intracellular  recordings  required  for  individual  identification,  the  physiological  results 
so  far  suggest  new  world  robberflies  to  employ  HI-,  V1-,  and  V2-like  cells  sensitive 
to  yaw,  combined  pitch/roll  and  roll,  respectively.  I  presented  some  of  those  results  in 
earlier  reports. 

As  the  provision  with  new  world  robberfies  for  experiments  in  the  UK  turned  out  to 
be  difficult  we  have  now  identified  a  species  common  at  the  English  south  coast, 
Dysmachus  trigonus  (Fig  6),  which  appears  from  early  May  until  end  of  August.  One 
of  my  postdocs,  Dr  Kit  Londgen,  was  funded  on  this  AFRL  grant  and  is  currently 
funded  by  the  joint  AFOSR  grant  between  Sean  Humbert  and  I,  has  succeeded  to 
perform  extracellular  recordings  from  Dysmachus,  which  is  slightly  smaller  in  size 
than  Calliphora.  He  obtained  the  first  LPTC  receptive  field  maps  from  this  robberfly 
species  and  also  data  on  the  temporal  response  properties  of  these  cells.  What  Kit  also 
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found  in  the  lobula  plate  of  Dysmachus  is  a  class  of  directional  selective  cells  which 
possess  comparatively  small  receptive  fields  located  slightly  below  the  eye  equator  in 
the  frontal  to  frontolateral  visual  field.  Cells  with  such  response  properties  are  not 
frequently  encountered  during  extracellular  recordings  from  Calliphora ,  even  though 
behavioural  experiments  on  the  elevation  sensitivity  of  compensatory  head  roll 
movements  suggest  a  neural  mechanism  that  specifically  enhances  the  response  to 
visual  motion  just  below  the  external  horizon  (see  below). 


Azimuth  [°j  Azimuth  [°] 

Figure  10:  Receptive  field  organization  of  spiking  lobula  plate  tangential  cells  in  Dysmachus 
trigonus.  Left:  The  local  preferred  directions  in  the  receptive  field  of  this  cell  depend  on  the 
azimuth  and  the  elevation  where  the  directional  tuning  is  assessed.  The  pattern  looks  similar 
to  that  observed  in  a  blowfly  dCH  cell  (OlO)  -  although  the  dCH  cell  in  blowflies  does  not 
generate  any  action  potentials.  Right:  The  distribution  of  local  preferred  directions  has 
similarities  to  that  of  a  blowfly  VI  receptive  field  (OlO)  -  here,  however,  the  receptive  field 
seems  to  be  more  confined  to  the  frontolateral  region  of  the  visual  field.  Insets  in  both  panels 
show  the  spiking  response  over  time  to  a  number  of  consecutive  motion  stimulus  cycles.  Note 
the  strongly  structured  response  where  phases  of  high  spike  rates  (during  movement  in  the 
preferred  direction)  are  followed  by  inhibition  (during  movement  in  the  null  direction)  which 
indicates  that  the  cell  receives  input  from  fully  opponent  elementary  movement  detectors  (e.g. 
rev.:  024). 
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Figure  11:  Partial  receptive  fields  of  (a)  the  Hl-cell  in  the  blowfly  Calliphora,  (b)  an  Hl-like 
cell  in  the  robberfly  Laphria,  and  (c)  an  Hl-like  cell  in  the  horsefly  Hematopota.  The  Hl-cell 
is  sensitive  to  yaw-rotations  while  being  inhibited  during  forwards  translation.  This  cell  was 
found  in  all  dipteran  flies  studied  so  far. 
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Figure  12:  Small-field  directional  selective  cells  in  the  lobula  plate  of  Dysmachus.  These  cells 
show  an  extremely  confined  receptive  field  of  no  more  than  20-30  degrees  in  diameter  located 
just  below  the  eye  equator  in  the  frontal  visual  field.  It  is  unlikely  that  the  localized  response 
of  these  cells  is  the  consequence  of  a  highly  nonlinear  integration  process  which  requires 
simultaneous  inputs  from  many  direction  selective  elements  before  a  spike  may  be  generated 
while  local  motion  stimuli  only  result  in  subthreshold  membrane  potential  changes.  The  cell 
sensitive  to  vertical  downward  motion  (right)  could  potentially  contribute  to  a  neural 
mechanism  that  stabilizes  the  position  of  the  external  horizon  just  below  the  animal’s  eye 
equator.  For  further  explanation,  see  text. 
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Figure  13:  Temporal  frequency  tuning  of  wide-field  and  small-field  LPTCs  in  Dysmachus. 
Moving  a  visual  grating  with  a  fixed  spatial  wavelength  at  different  velocities  shows  that 
small-field  cells  possess  a  significantly  narrower  temporal  frequency  tuning  than  wide-field 
cells.  Together  with  the  marked  difference  in  receptive  field  size  (Fig  11),  this  suggests  small- 
field  cells  may  be  involved  in  small  target  detection  as  opposed  to  the  estimation  of  self- 
motion. 

The  conclusions  of  our  studies  on  robberfly  LPTCs  so  far  are:  (i)  Spiking  wide-field 
LPTCs  in  robberflies  most  likely  serve  a  similar  function  like  blowfly  wide-field 
LPTCs  do  -  i.e.  the  estimation  of  self-motion  parameters  and  (ii)  small-field  LPTCs 
may  either  be  adapted  to  detect  small  targets  or  may  be  involved  in  horizon  detection 
in  case  they  have  vertical  motion  preferences.  The  former  function,  small  target 
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detection,  may  be  similar  to  that  of  figure  detection  cells  (FD-cells,  Egelhaaf  1985)  in 
the  blowfly.  Further  experiments  will  be  required  to  study  their  response  properties  in 
more  detail.  An  alternative  interpretation,  at  least  for  cells  sensitive  to  vertical  motion, 
would  be  that  they  support  a  horizon  detection  mechanism  reported  in  the  context  of 
gaze  stabilization  in  blowflies  (see  above).  Hengstenberg  proposed  such  mechanism 
based  on  earlier  work  on  gaze  stabilization  which  was  further  substantiated  by  an 
increased  gain  of  compensatory  head  roll  just  below  the  eye  equator  (Fig  14,  A6). 
This  increased  gain  could  not  be  explained  by  the  receptive  field  organization  of  the 
VS-cells,  a  subset  of  wide-field  LPTCs  indicating  rotations  around  the  fly’s  horizontal 
body  axes  (01,  03).  Although  small-field  LPTCs  similar  to  those  encountered  in  the 
robberfly  have  not  been  reported  in  blowflies,  some  blowfly  neck  motor  neurons 
show  receptive  field  with  similarly  small  receptive  fields  (026). 
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Figure  14:  Gain  of  compensatory  head 
roll  (filled  triangles)  and  quantitatively 
estimated  contributions  of  groups  of  VS 
cells  (filled  squares)  in  Calliphora.  The 
head  roll  sensitivity  was  determined  by 
oscillating  a  narrow  horizontal  bar 
covering  90  degree  in  azimuth  of  the 
lateral  visual  field  at  0.63  Hz  and  at  an 
amplitude  of  +/-  15  degrees.  The  resulting 
response  gain  is  plotted  as  a  function  of 
elevation  (all  three  panels).  The  same 
stimulus  was  projected  into  the  receptive 
fields  of  three  groups  of  VS-cell  to 
estimate  the  neurons’  responses.  Although 
the  combined  and  normalized  response 
estimate  of  VS4-VS7  closely  follows  head 
roll  sensitivity ,  at  an  elevation  of  -15 
degree  there  is  a  significant  difference 
between  the  behavioural  data  and  the 
estimated  neural  responses  (middle  panel). 
This  result  suggests  an  additional  neural 
mechanism  contributing  to  head  roll 
sensitivity  just  below  the  eye  equator. 
(Data  from  A6.) 


State-dependent  processing  of  optic  flow. 

Information  processing  in  the  nervous  system  is  a  significant  cost  factor  in  the  overall 
metabolic  energy  budget  of  an  animal.  Sensory  cells  in  particular  may  consume  a 
considerable  amount  of  energy  in  an  attempt  to  reduce  response  delays  at  the  expense 
of  low  input  resistances  and  high  ion  currents  across  the  cell  membrane.  The  same  is 
true  for  large  integrating  cells  such  as  the  LPTCs.  These  cells  have  a  comparatively 
low  input  resistance  resulting  in  a  resting  potential  of  only  around  -50  mV  compared 
to  -70  mV  in  other  nerve  cells.  The  low  input  resistance  reduces  the  membrane  time 
constant  and  speeds  up  the  neurons’  responses.  On  the  other  hand  a  low  input 
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resistance  due  to  many  open  ion  channels  requires  more  energy  to  maintain  the 
required  concentration  gradients  of  ion  species  involved  in  signalling  such  as  sodium, 
potassium,  calcium  and  chloride.  To  a  certain  degree  this  is  a  necessary  investment  as 
it  enables  short  response  delays  to  visual  stimulation.  At  the  sensory  and  central 
processing  levels  this  results  is  a  trade-off  between  higher  bandwidth  and  higher 
energy  consumption  (Niven  and  Laughlin  2008). 

Such  trade-off  concerns  not  only  the  immediate  costs  of  information  transfer  in  the 
nervous  system.  The  consequences  of  inaccurate  sensory  information  processing  may 
be  by  far  more  severe.  Erroneous  sensory  signals  may  result  in  massive  waist  of 
metabolic  energy  due  to  imprecise  flight  manoeuvres  executed  by  the  power  muscles 
of  the  flight  motor,  which  arguably  consumes  even  more  energy  than  the  nervous 
system.  Altogether,  the  most  appropriate  strategy  would  be  to  safe  energy  when  the 
animal  faces  a  limited  dynamic  stimulus  range  and  to  invest  more  energy  only  when 
the  stimulus  dynamics  increase  -  which  normally  is  the  case  when  the  animal 
performs  any  kind  of  locomotor  activity. 


Two  years  ago  Kit  Longden  applied  an  octopamine  agonist  chlordimephorm  (CDM)  - 
a  drug  known  to  push  the  nervous  system  of  insects  into  a  fictive  locomotor  state  -  to 
the  blowfly  haemolymph  while  recording  from  spiking  LPTCs.  The  immediate  effect 
was  that  the  cells  increased  their  spontaneous  spike  rate,  responded  faster  to  visual 
stimuli  and  transmitted  more  information  about  directional  motion  (031).  He 
continued  his  work  to  publish  another  paper  where  he  also  studied  the  impact  of  CDM 
on  the  temporal  response  characteristics  of  LPTCs  (036).  Several  other  labs  also 
started  working  on  the  relationship  between  locomotor  state  and  visual  processing 
which  has  now  become  an  established  research  area  in  insect  motion  vision.  We  also 
published  a  dispatch  in  Current  Biology  featuring  work  on  this  topic  (039).  In  the 
following  I  will  present  some  of  the  more  recent  results  Kit  presented  in  a  dedicated 
symposium  on  state-dependent  information  processing  at  the  International  Society  of 
Neuroethology  in  Maryland,  USA.  Finally,  I  will  summarize  the  findings  and  will 
discuss  the  relevance  of  the  work  for  the  design  of  micro  air  vehicles. 


100  - 


. 


Yaw 

velocity 

m 


50 

-50 


Times  i  s 


Figure  15:  Simultaneously  monitoring  neural  activity  and  walking  in  tethered  blowflies.  Left: 
A  blowfly  is  walking  on  an  air-suspended  styrofoam  ball  the  rotations  of  which  are  measured 
with  photoelectric  devices  (optical  mice)  and  converted  into  a  yaw  signal.  At  the  same  time 
the  spiking  activity  of  the  H2-cell  is  recorded  using  tungsten  electrodes.  Right:  The  upper 
panel  shows  spiking  activity  (1st  trace)  when  the  fly  is  at  rest  as  indicated  by  the  zero  yaw 
velocity  (2nd  trace).  The  lower  panel  shows  increased  spiking  activity  (1st  trace)  when  the  fly 
spontaneously  starts  to  move  (2nd  trace).  (Longden,  work  in  progress). 
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Rather  than  staying  with  the  drug  CDM,  Kit  has  now  set  up  an  experimental  rig  which 
allows  him  to  monitor  the  walking  behaviour  of  Calliphora  while  recording  the  neural 
activity  of  spiking  LPTCs  (Figs.  15,  16).  He  also  applied  octopamine,  the  neuro¬ 
modulator  involved  in  changing  the  neuronal  activity  and  switching  the  animal’s 
metabolic  pathways  in  preparation  for  flight.  Additionally,  he  used  an  octopamine 
antagonist  in  an  attempt  to  reduce  the  effect  walking  has  on  processing  visual  motion 
in  LPTCs  (Fig.  17). 
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Figure  16:  Using  the  setup  shown  in  the  previous  figure,  the  fly  is  presented  with  visual 
stimuli  which  include  two  different  components.  Upper  left  panel:  A  visual  grating  is  first 
moved  in  the  anti-preferred  direction  of  a  recorded  H2-cell.  This  stimulus  inhibits  spiking  in 
the  cell.  Immediately  after  3  s  of  anti-preferred  direction  movement,  the  stimulus  is  moved 
into  the  cell’s  preferred  direction  for  500  ms,  followed  by  the  presentation  of  a  blank  screen 
of  average  brightness.  The  response,  convolved  with  a  Gaussian  filter,  is  shown  on  top  of  the 
stimulus  trace.  Upper  right  panel:  The  spontaneous  spike  rate  (ssr)  -  i.e.  without  the  motion 
stimulus  -  is  also  recorded  and  categorized  according  to  a  threshold  criterion  related  to  the 
current  walking  speed  of  the  animal  which  is  either  slow  or  fast.  The  ssr  in  fast  walking 
animals  is  significantly  higher  than  in  slowly  walking  animal.  Lower  right  panel:  The 
temporal  frequency  tuning  of  the  H2-cell.  The  response  minus  ssr  is  plotted  against  different 
temporal  frequencies  in  slowly  (black)  and  fast  (orange)  walking  flies.  At  high  temporal 
frequencies  fast  moving  flies  show  a  stronger  response  than  slowly  moving  flies  which 
suggests  that  walking  increases  the  bandwidth  of  the  cell  due  to  reduced  motion  adaptation. 
(Longden,  work  in  progress). 
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Figure  17:  The  same  protocol  as  described  in  figure  legend  16,  applied  to  two  different 
experimental  conditions.  Solid  curves  indicate  results  obtained  under  the  influence  of 
epinastine,  an  octopamine  antagonist.  Dotted  curves  were  obtained  without  epinastine 
(controls).  Black  and  read  curves  give  data  gathered  during  slow  and  fast  walking, 
respectively.  While  epinastine  reduces  the  impact  of  walking  on  the  ssr  (upper  right  panel),  its 
effect  on  the  temporal  tuning  curve  is  comparatively  small.  Octopamine,  thus,  seems  to  be 
involved  in  increasing  the  ssr  and  has  a  small  effect  on  the  cell’s  changes  in  temporal 
frequency  tuning  during  walking  (lower  right  panel).  (Longden,  work  in  progress). 

Our  results  over  the  last  2  years  support  the  following  conclusions  regarding  the 
mechanisms  in  place  to  produce  state-dependent  changes  in  visual  motion  processing: 


•  CDM  increases  the  spontaneous  activity  (ssr),  reduces  the  response  delay,  and 
increases  the  dynamic  output  range  of  the  cell  by  an  enhanced  negative 
signalling  range. 

•  Locomotion  on  its  own  increases  the  spontaneous  activity  (ssr)  significantly, 
has  no  direct  effect  on  the  temporal  frequency  tuning  but  affects  motion 
adaptation  which  increases  the  sensitivity  to  higher  temporal  frequencies. 

•  Locomotion  in  combination  with  motion  adaptation  reduces  the  response 
delay  and  increases  the  gain  for  all  temporal  frequencies  in  a  physiological 
time  window  of  20-80  ms  after  stimulus  presentation. 

•  Octopamine  antagonist  reduces  both  the  ssr  and  the  response  gain,  and 
increases  the  response  delay. 

Altogether  the  effects  suggest  that  blowflies  save  energy  during  rest  by  lowering  the 
spontaneous  spike  rate  and  response  gain  to  visual  motion.  Only  when  switching  to  an 
active  locomotor  state  the  response  delays  are  reduced  and  the  bandwidth  of  the  cells 
is  increased.  Although  octopamine  is  involved  in  state-dependent  motion  processing 
the  effects  may  be  mediated  through  other  mechanisms,  for  instance  motion 
adaptation.  Any  impact  on  the  temporal  frequency  tuning  of  the  cell,  therefore,  may 
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be  due  to  neuro-modulator  action  in  combination  with  an  expanded  stimulus/velocity 
distribution  during  locomotion. 

The  significance  of  the  neural  mechanisms  underlying  state-dependent  visual  motion 
processing  is  obvious  from  the  design  perspective  regarding  the  development  of 
autonomous  micro  air  vehicles.  Size  and  payload  limitations  do  not  permit  the  use  of 
power-hungry  high  throughput  floating  point  devices  for  the  processing  and 
integration  of  sensor  signals  including  their  transformation  into  actuator  commands. 
This  is  where,  firstly,  task-specific  integration  of  local  motion  information  becomes 
particularly  efficient  which  results  in  an  output  signal  that  can  be  applied  immediately 
to  control  state  changes.  And  secondly  -  assuming  analogue  processing  devices  in 
VLSI  technology  -  where  continuous  adjustments  of  input  and  output  bandwidth  may 
mitigate  the  problems  of  power  consumption  in  two  ways:  (i)  Saving  energy  when  the 
input  bandwidth  is  low  and  (ii)  implementing  circuits  in  low-power  VLSI  in  the  first 
place. 


Fly-robot  interface  far  studying  multisensory  integration. 

This  ongoing  work  aims  to  establish  a  closed-loop  platform  that  allows  us  to  record 
the  activity  of  a  spiking  LPTC,  the  HI -cell,  in  a  fly  that  is  mounted  on  a  two-wheeled 
robot.  The  recorded  spiking  activity  is  converted  into  a  command  signal  applied  to  the 
motors  driving  the  robot  wheels  so  that  the  fly  does  not  collide  with  any  obstacles.  A 
simplified  version  of  such  a  closed-loop  system  has  been  fully  characterized  and  used 
to  assess  the  performance  of  the  HI -cell  as  a  motion  vision  sensor  under  different 
feedback  control  laws  in  an  image  stabilization  task  (038,  C4,  C5). 

The  dynamics  of  the  robot  are  chosen  so  that  several  sensory  systems  will  be 
stimulated,  including  the  antennae,  halteres  and  compound  eyes.  By  systematically 
disabling  individual  sensor  systems  we  will  test  whether  sensory  modalities  other  than 
motion  vision  impact  on  the  activity  of  the  HI -cell  and  other  spiking  LPTCs  while  the 
robot  is  moving  under  closed-loop  conditions.  From  previous  open-loop  studies  we 
already  know  that  the  VI -cell,  for  instance,  reflects  the  activity  of  other  sensory 
systems,  e.g.  the  ocelli.  One  of  the  challenges  of  this  project  has  been  to  miniaturize 
electrophysiological  equipment  so  it  does  not  add  to  much  inertia  to  the  closed-loop 
robotic  platform  while  still  enabling  stable  extracellular  recordings.  After  we  have 
now  succeeded  to  design  a  miniaturized  and  stable  recording  module  we  will  soon  be 
able  to  perform  the  first  experiments  with  the  fly  steering  the  robot. 

Related  to  this  project  is  our  attempt  to  develop  a  miniaturized  extracellular  amplifier 
small  enough  to  fit  into  the  head  capsule  of  a  blowfly  -  a  project  funded  by  the  HFSP 
trust.  The  amplifier  has  already  been  successfully  produced  and  tested  (C3).  We  are 
still  working  on  electrode  chip  that  provides  the  front  end  to  the  systems.  It  will 
consist  of  3  differential  recording  channels  to  measure  the  signals  in  spiking  LPTCs  in 
freely  or  semi-freely  moving  flies  the  trajectory  of  which  will  be  monitored  by  high 
speed  video  cameras.  The  video  footage  will  be  used  to  reconstruct  the  fly’s  self- 
motion  components  which  can  then  be  correlated  with  the  neural  activity. 
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In  both  these  projects  we  should  also  be  able  to  retrieve  any  traces  of  efference  copies 
(forward  models)  the  fly  might  be  using  to  improve  its  sensoimotor  control 
performance. 


2.4.  Modelling  of  multisensory  motor  control  design 

If  the  performance  of  a  control  system  is  readily  captured  by  a  set  of  transfer 
functions  which  are  normally  sufficient  to  specify  control  architectures,  why  do  we 
bother  about  (a)  neuronal  mechanisms  and  (b)  mechanistic  or  biomechanical  models? 
Why  should  we  not  just  go  ahead  and  implement  on  an  MAV  the  system  properties 
described  in  terms  of  phenomenological  models? 

There  are  a  couple  of  answers  to  this  question  which  are  often  neglected  when  it 
comes  to  implementation.  One  of  the  major  points  is  that  the  nervous  system  in 
insects  with  its  comparatively  limited  capacity  in  terms  of  number  of  neurons  has  to 
serve  several  high  specification  control  tasks  simultaneously.  Although  the  life  of  an 
insect  is  concerned  more  or  less  only  with  two  tasks  -  feeding  and  mating  -  the 
enabling  motor  programmes  are  quite  diverse  as  they  do  not  simply  produce  one  fixed 
behavioural  sequence  but  rather  accomplish  the  tasks  in  a  context-dependent  way. 
Foraging  flights,  where  navigating  to  a  rewarding  feeder  is  the  primary  objective,  may 
be  interrupted  by  collision  avoidance  manoeuvres  or  external  disturbances  due  to 
gusts  of  wind  which  need  to  be  compensated  for  to  maintain  aerodynamic  stability. 
Often  this  means  that  incoming  sensory  information  has  to  be  processed  and  fed  back 
into  the  flight  controller  in  very  different  ways  using  one  and  the  same  final  neuronal 
pathway  to  the  motor  systems.  Another  factor  that  has  been  pointed  out  recently  is  the 
limited  amount  of  energy  available  to  the  insect  for  both  powering  up  the  muscles  and 
providing  the  nervous  system  (see  above).  And  finally,  the  way  in  which  insects 
acquire  information  and  convert  sensor  readings  into  actuator  commands  depends  on 
the  structural  and  dynamic  properties  of  the  sensors  and  the  functional  organization  as 
well  as  the  dynamics  of  the  motor  systems  they  control.  This  is  probably  true  in 
robotics,  too,  only  that  the  task  space  of  most  robots  is  comparatively  well  specified 
which  often  allows  for  a  simpler  mechanical  and  control  design.  Insects  use  the  same 
hardware  to  solve  very  different  tasks  and  need  to  modulate  their  sensorimotor 
transformation  depending  on  which  task  is  currently  being  controlled.  Switching,  for 
instance,  between  olfactory-driven  navigation  and  collision  avoidance,  may  be 
required  within  a  fraction  of  a  second. 

The  integration  of  inner-loop  reflex  behaviour  and  outer-loop  voluntary  -  or  task 
related  -  behaviour  imposes  a  fundamental  constraint  on  the  way  in  which  different 
modelling  approaches  may  be  exploited.  So  far  we  applied  mostly  linear  systems 
approaches  to  work  out  the  way  in  which  various  sensory  modalities  are  combined  in 
fly  gaze  stabilization  (see  above,  037,  041).  In  collaboration  with  Dr  Reiko  Tanaka, 
Bioengineering,  Imperial  College,  we  develop  a  closed-loop  simulation  platform  that 
emulates  gaze  stabilization  in  a  linear  control  engineering  framework  including 
systems  identification  methods.  But  it  became  obvious  already  that  several 
experimental  results  are  not  easily  explained  by  the  linear  combination  of  transfer 
functions  (see  above  -  compound  eye/ocellar  interactions).  We  therefore  combine 
different  approaches  always  starting  with  the  most  parsimonious  model,  i.e.  a  linear 
one.  In  cases  where  we  do  not  capture  the  overall  behaviour  of  the  system  we  add 
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complexity  by  including  static  nonlinearity  to  mechanistic  models  (see  above). 
Finally,  as  mentioned  earlier,  biomechanical  models  will  augment  our  modelling 
efforts  in  particular  when  it  comes  to  the  connection  between  sensing  and  actuation 
where,  in  biology,  much  of  the  computational  workload  is  reduced  by  the  choice  of 
the  mechanical  design  and  material  properties. 

Biomechanical  and  mechanistic  neuronal  modelling  will  be  important  to  identify 
those  physiological  mechanisms  which  make  biological  systems  special  in  terms  of 
their  performance.  As  mentioned  earlier  -  the  most  interesting  question  is  as  to  how 
different  non-linear  systems  manage  to  produce  a  system’s  output  that  looks  by  and 
large  linear.  In  biology,  non-linearities  may  as  well  be  beneficial,  despite  all  the  risks 
they  bear  from  the  perspective  of  a  control  engineer. 


3.  Conclusion: 

Besides  the  academic  outputs  produced  during  the  period  of  funding  -  and  probably 
beyond  -  one  of  the  most  important  outcomes  over  the  last  couple  of  year  has  been  a 
clear  research  agenda  for  my  lab.  While  collaborating  with  the  groups  of  Graham 
Taylor  and  Sean  Humbert  my  research  has  shaped  into  a  system  neuroscience 
approach  that  combines  quantitative  behaviour,  electrophysiology,  functional 
anatomy  and  modelling  to  advance  our  understanding  of  biological  control  design  - 
with  a  strong  emphasis  on  the  underlying  neuronal  mechanisms.  In  particular  the 
interactions  with  Sean  Humbert  have  resulted  in  several  tangible  outputs,  e.g.  the 
proof  of  concept  study  showing  that  LPTC  receptive  fields  can  be  successfully  used  to 
control  orientation,  proximity  and  forward  speed  of  a  quadropter  (035)  or  the 
development  of  an  ocellar  senor  (C6).  Future  interactions  between  my  group  and  the 
groups  of  my  collaborators  will  aim  to  further  discover  and  exploit  biological  design 
principles  in  the  context  of  control  and  navigation  of  autonomous  robotic  systems. 
The  emphasis  will  be  to  advance  autonomy  and  manoeuvrability  by  multisensor 
fusion,  dynamic  range  fractioning  and  the  integration  of  inner-  and  outer  loop  control. 


Holger  G  Krapp 


London,  13.  August  2012 
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